Wearable 3d augmented reality display with variable focus and/or object recognition

ABSTRACT

A wearable  3 D augmented reality display and method, which may include 3D integral imaging optics.

RELATED APPLICATIONS

This application of claims the benefit of priority of U.S. ProvisionalApplication No. 61/948,226, filed on Mar. 5, 2014, the entire contentsof which application(s) are incorporated herein by reference.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under Grant Nos.14-22653 and 14-22179 awarded by the National Science Foundation. Thegovernment has certain rights in the invention.

FIELD OF THE INVENTION

The present invention relates generally to a wearable 3D augmentedreality display, and more particularly, but not exclusively, to awearable 3D augmented reality display comprising 3D integral imaging(InI) optics with optional variable focus and/or object recognition.

BACKGROUND OF THE INVENTION

An augmented reality (AR) display, which allows overlaying 2D or 3Ddigital information on a person's real-world view, has long beenportrayed as a transformative technology to redefine the way we perceiveand interact with digital information. Although several types of ARdisplay devices have been explored, a desired form of AR displays is alightweight optical see-through head-mounted display (OST-HMD), whichenables optical superposition of digital information onto the directview of the physical world and maintains see-through vision to the realworld. With the rapidly increased bandwidth of wireless networks, theminiaturization of electronics, and the prevailing cloud computing, oneof the current challenges is to realize an unobtrusive AR display thatintegrates the functions of OST-HMDs, smart phones, and mobile computingwithin the volume of a pair of eyeglasses.

Such an AR display, if available, will have the potential torevolutionize many fields of practice and penetrate through the fabricof life, including medical, defense and security, manufacturing,transportation, education and entertainment fields. For example, inmedicine AR technology may enable a physician to see CT images of apatient superimposed onto the patient's abdomen while performingsurgery; in mobile computing it can allow a tourist to access reviews ofrestaurants in his or her sight while walking on the street; in militarytraining it can allow fighters to be effectively trained in environmentsthat blend 3D virtual objects into live training environments.

Typically, the most critical barriers of AR technology are defined bythe displays. The lack of high-performance, compact and low-cost ARdisplays limits the ability to explore the full range of benefitspotentially offered by AR technology. In recent years a significantresearch and market drive has been toward overcoming the cumbersome,helmet-like form factor of OST-HMD systems, primarily focusing onachieving compact and lightweight form factors. Several opticaltechnologies have been explored, resulting in significant advances inOST-HMDs. For instance, the well-advertised Google Glass® is a verycompact, lightweight (˜36 grams), monocular OST-HMD, providing thebenefits of encumbrance-free instant access to digital information.Although it has demonstrated a promising and exciting future prospect ofAR displays, the current version of Google Glass® has a very narrow FOV(approximately 15° FOV diagonally) with an image resolution of 640×360pixels. It offers limited ability to effectively augment the real-worldview in many applications.

Despite such promises a number of problems remain with existingOST-HMD's, such as visual discomfort of AR displays. Thus, it would bean advance in the art to provide OST-HMD's which provide increasedvisual comfort, while achieving low-cost, high-performance, lightweight,and true 3D OST-HMD systems.

SUMMARY OF THE INVENTION

In one of its aspects the present invention may provide a 3D augmentedreality display having a microdisplay for providing a virtual 3D imagefor display to a user. For example, the optical approach of the presentinvention may uniquely combine the optical paths of an AR display systemwith that of a micro-InI subsystem to provide a 3D lightfield opticalsource. This approach offers the potential to achieve an AR displayinvulnerable to the accommodation-convergence discrepancy problem.Benefiting from freeform optical technology, the approach can alsocreate a lightweight and compact OST-HMD solution.

In this regard, in one exemplary configuration of the present invention,display optics may be provided to receive optical radiation from themicrodisplay and may be configured to create a 3D lightfield, that is, atrue optically reconstructed 3D real or virtual object from the receivedradiation. (As used herein the term “3D lightfield” is defined to meanthe radiation field of a 3D scene comprising a collection of light raysappearing to be emitted by the 3D scene to create the perception of a 3Dscene.) An eyepiece in optical communication with the display optics mayalso be included, with the eyepiece configured to receive the 3Dlightfield from the display optics and deliver the received radiation toan exit pupil of the system to provide a virtual display path. Theeyepiece may include a selected surface configured to receive the 3Dlightfield from the display optics and reflect the received radiation toan exit pupil of the system to provide a virtual display path. Theselected surface may also be configured to receive optical radiationfrom a source other than the microdisplay and to transmit such opticalradiation to the exit pupil to provide a see-through optical path. Theeyepiece may include a freeform prism shape. In one exemplaryconfiguration the display optics may include integral imaging optics.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary and the following detailed description of theexemplary embodiments of the present invention may be further understoodwhen read in conjunction with the appended drawings, in which:

FIGS. 1A to 1C schematically illustrate accommodation-convergence cuesin a monocular AR display (FIG. 1A); a binocular display (FIG. 1B); and,viewing a real object (FIG. 1C);

FIG. 2 schematically illustrates a block diagram of an exemplary3D-OST-HMD system in accordance with the present invention, comprising amicroscopic integral imaging (InI) unit, see-through optics, andeyepiece;

FIG. 3 schematically illustrates a diagram of a microscopic InI unit forcreating a 3D lightfield of a 3D scene for use in devices and methods ofthe present invention;

FIG. 4 schematically illustrates a diagram of an alternative exemplarymicroscopic InI (micro-InI) unit in accordance with the presentinvention for creating a 3D lightfield of a 3D scene where the virtuallightfield is telecentric;

FIG. 5 schematically illustrates a diagram of an exemplary head-worn 3Dintegral imaging display system in accordance with the presentinvention, which integrates a micro-InI unit and conventional eyepieceoptics for creating a virtual lightfield of a 3D scene;

FIGS. 6A to 6C schematically illustrate an exemplary design of a 3Daugmented reality optical see-through HMD in accordance with the presentinvention using freeform optical technology, in which FIG. 6Aillustrates an exemplary freeform eyepiece for 3D lightfield display,FIG. 6B illustrates an exemplary freeform corrector lens to correctviewing axis deviations and aberrations, and FIG. 6C illustrates anintegrated optical layout and raytracing;

FIG. 6D schematically illustrates an exemplary design of a 3D augmentedreality optical see-through HMD in accordance with the present inventionincluding a vari-focal element;

FIG. 6E schematically illustrates an exemplary design of a 3D augmentedoptical see-through HMD in accordance with the present inventionincluding 3D and/or 2D object recognition;

FIG. 7 schematically illustrates an exemplary micro-InI module andeyepiece in accordance with the present invention;

FIG. 8 illustrates an exemplary prototype of a microdisplay, microlensarray, 3D scene reconstructed by micro-InI, and a free form eyepiece inaccordance with the present invention;

FIG. 9 illustrates the experimental “3D” image used in a particulardemonstration of the invention; and

FIGS. 10A to 10D demonstrate images captured by a digital camera placedat the eyepiece of the prototype of FIG. 8 where the camera was focusedat 4 m (FIG. 10A), 30 cm (FIG. 10B), shifted to the left side of theexit pupil (FIG. 10C), and shifted to the right side of the exit pupil(FIG. 10D).

DETAILED DESCRIPTION OF THE INVENTION

Despite current commercial development of HMDs, very limited effortshave been made to address the challenge of minimizing visual discomfortof AR displays, which is a critical concern in applications requiring anextended period of use. One of the key factors causing visual discomfortis the accommodation-convergence discrepancy between the displayeddigital information and the real-world scene, which is a fundamentalproblem inherent to most of the existing AR displays. The accommodationcue refers to the focus action of the eye where ciliary muscles changethe refractive power of the crystalline lens and therefore minimize theamount of blur for the fixated depth of the scene. Associated with eyeaccommodation change is the retinal image blur cue which refers to theimage blurring effect varying with the distance from the eye's fixationpoint to the points nearer or further away. The accommodation andretinal image blurring effects together are known as focus cues. Theconvergence cue refers to the rotation action of the eyes to bring thevisual axes inward or outward to intersect at a 3D object of interest atnear or far distances.

The accommodation-convergence mismatch problem stems from the fact thatthe image source in most of the existing AR displays is a 2D flatsurface located at a fixed distance from the eye. Consequently, thistype of AR display lacks the ability to render correct focus cues fordigital information that is to be overlaid over real objects located atdistances other than the 2D image source. It causes the following threeaccommodation-convergence conflict. (1) There exists a mismatch ofaccommodation cues between the 2D image plane and the real-world scene(FIG. 1A). The eye is cued to accommodate at the 2D image plane forviewing the augmented information while the eye is concurrently cued toaccommodate and converge at the depth of a real 3D object onto which thedigital information is overlaid. The distance gap between the displayplane and real-world objects can be easily beyond what the human visualsystem (HVS) can accommodate simultaneously. A simple example is the useof an AR display for driving assistance where the eyes need toconstantly switch attention between the AR display and real-worldobjects spanning from near (e.g. dashboard) to far (e.g. road signs).(2) In a binocular stereoscopic display, by rendering a pair ofstereoscopic images with binocular disparities, the augmentedinformation may be rendered to appear at a different distance from the2D display surface (FIG. 1B). When viewing augmented information, theeye is cued to accommodate at the 2D display surface to bring thevirtual display in focus but at the same time the eye is forced toconverge at the depth dictated by the binocular disparity to fuse thestereoscopic pair. In viewing a natural scene (FIG. 1C), the eyeconvergence depth coincides with the accommodation depth and objects atdepths other than the object of interest are seen blurred. (3) Syntheticobjects rendered via stereoscopic images, regardless of their rendereddistance from the user, are seen all in focus if the viewer focuses onthe image plane, or are seen all blurred if the user accommodates atdistances other than the image plane. The retinal image blur of adisplayed scene does not vary with the distances from an eye fixationpoint to other points at different depths in the simulated scene. In anutshell, the incorrect focus cues may contribute to issues in viewingstereoscopic displays, such as distorted depth perception, diplopicvision, visual discomfort and fatigue, and degradation in oculomotorresponse.

In one of its aspects the present invention relates to a novel approachto OST-HMD designs by combining 3D lightfield creation technology andfreeform optical technology. 3D lightfield creation technology of thepresent invention reconstructs the radiation field of a 3D scene bycreating a collection of light rays appearing to be emitted by the 3Dscene and creating the perception of a 3D scene. Thus, as used hereinthe term “3D lightfield” is defined to mean the radiation field of a 3Dscene comprising a collection of light rays appearing to be emitted bythe 3D scene to create the perception of a 3D scene. The reconstructed3D scene creates a 3D image source for HMD viewing optics, which enablesthe replacement of a typical 2D display surface with a 3D source andthus potentially overcomes the accommodation-convergence discrepancyproblem. Any optical system capable of generating a 3D lightfield may beused in the devices and methods of the present invention. For instance,one exemplary configuration of the present invention uses micro integralimaging (micro-InI) optics for creating a full-parallax 3D lightfield tooptically create the perception of the 3D scene. (Persons skilled in theart will be aware that Integral imaging (InI) is a multi-view imagingand display technique that captures or displays the light fields of a 3Dscene by utilizing an array of pinholes, lenses or microlenses. In thecase of being a display technique, a microlens array in combination witha display device, which provides a set of elemental images each havinginformation of a different perspective of the 3D scene. The microlensarray in combination with the display device renders ray bundles emittedby different pixels of the display device, and these ray bundles fromdifferent pixels intersect and optically create the perception of a 3Dpoint that appears to emit light and occupy the 3D space. This methodallows the reconstruction of a true 3D image of the 3D scene with fullparallax information in all directions.) Other optical system capable ofgenerating a 3D lightfield which may be used with the present inventioninclude, but not limited to, holographic display (M. Lucente,“Interactive three-dimensional holographic displays: seeing the futurein depth,” Computer Graphics, 31(2), pp. 63-67, 1997; P. A. Blanche, etal, “Holographic three-dimensional telepresence using large-areaphotorefractive polymer”, Nature, 468, 80-83, November 2010),multi-layer computational lightfield display (G. Wetzstein et al.,“Tensor Displays: Compressive light field synthesis using multilayerdisplays with directional backlighting,” ACM Transactions on Graphics,31(4), 2012.), and volumetric displays (Blundell, B. G., and Schwarz, A.J., “The classification of volumetric display systems: characteristicsand predictability of the image space,” IEEE Transaction onVisualization and Computer Graphics, 8(1), pp. 66-75, 2002. J. Y. Son,W. H. Son, S. K. Kim, K. H. Lee, B. Javidi, “Three-Dimensional Imagingfor Creating Real-World-Like Environments,” Proceedings of IEEE Journal,Vol. 101, issue 1, pp. 190-205, January 2013.).

A micro-InI system has the potential of achieving full-parallax 3Dobject reconstruction and visualization in a very compact form factorsuitable for a wearable system. It can dramatically alleviate most ofthe limitations in a conventional autostereoscopic InI display due tothe benefit of well-constrained viewing positions and can be effectivelyutilized for addressing the accommodation-convergence discrepancyproblem in conventional HMD systems. The micro-InI unit can reconstructa miniature 3D scene through the intersection of propagated ray conesfrom a large number of recorded perspective images of a 3D scene. Bytaking advantage of the freeform optical technology, the approach of thepresent invention can result in a compact, lightweight, goggle-style ARdisplay that is potentially less vulnerable to theaccommodation-convergence discrepancy problem and visual fatigue.Responding to the accommodation-convergence discrepancy problem ofexisting AR displays, we developed an AR display technology with theability to render the true lightfield of a 3D scene reconstructedoptically and thus accurate focus cues for digital information placedacross a large depth range.

The challenges of creating a lightweight and compact OST-HMD solution,invulnerable to the accommodation-convergence discrepancy problem, areto address two cornerstone issues. The first is to provide thecapability of displaying a 3D scene with correctly rendered focus cuesfor a scene's intended distance correlated with the eye convergencedepth in an AR display, rather than on a fixed-distance 2D plane. Thesecond is to create an optical design of an eyepiece with a form factoras compelling as a pair of eyeglasses.

A block diagram of a 3D OST-HMD system in accordance with the presentinvention is illustrated in FIG. 2. It includes three principalsubsystems: a lightfield creation module (“3D Lightfield CreationModule”) reproducing the full-parallax lightfields of a 3D scene seenfrom constrained viewing zones; an eyepiece relaying the reconstructed3D lightfields into a viewer's eye; and a see-through system(“See-through Optics”) optically enabling a non-obtrusive view of thereal world scene.

In one of its aspects, the present invention provides an innovativeOST-HMD system that integrates the 3D micro-InI method for full-parallax3D scene optical visualization with freeform optical technology forOST-HMD viewing optics. This approach enables the development of acompact 3D InI optical see-through HMD (InI-OST-HMD) with full-parallaxlightfield rendering capability, which is anticipated to overcome thepersisting accommodation-convergence discrepancy problem and tosubstantially reduce visual discomfort and fatigue experiences of users.

Full-parallax lightfield creation method. An important step to addressthe accommodation-convergence discrepancy problem is to provide thecapability of correctly rendering the focus cues of digital informationregardless of its distance to the viewer, rather than rendering digitalinformation on a fixed-distance 2D surface. Among the differentnon-stereoscopic display methods, we chose to use an InI method thatallows the reconstruction of the full-parallax lightfields of a 3D sceneappearing to be emitted by a 3D scene seen from constrained orunconstrained viewing zones. Compared with all other techniques, an InItechnique requires a minimal amount of hardware complexity, which makesit possible to integrate it with an OST-HMD optical system and create awearable true 3D AR display.

FIG. 3 schematically illustrates an exemplary micro-InI unit 300. A setof 2D elemental images 301, each representing a different perspective ofa 3D scene, are displayed on a high-resolution microdisplay 310. Througha microlens array (MLA) 320, each elemental image 301 works as aspatially-incoherent object and the conical ray bundles emitted by thepixels in the elemental images 301 intersect and integrally create theperception of a 3D scene, in which objects appear to be located alongthe surface AOB having a depth range Z₀ at a reference plane, forexample, to provide the appearance to emit light and occupy the 3Dspace. The microlens array may be placed a distance “g” from themicrodisplay 310 to create either a virtual or a real 3D scene. Themicro-InI unit 300 allows the optical reconstruction of a 3D surfaceshape with full parallax information. It should be noted that anInI-based 3D display operates fundamentally differently from multi-viewstereoscopic systems where a lenticular sheet functions as a spatialde-multiplexer to select appropriate discrete left-eye and right-eyeplanar views of a scene dependent on viewer positions. Such multi-viewsystems produce a defined number of binocular views typically withhorizontal parallax only and may continue to suffer from convergenceaccommodation conflict.

FIG. 4 schematically illustrates an alternative configuration of amicro-InI unit 400 in accordance with the present invention that createsa telecentric 3D lightfield of a 3D scene at surface AOB. A primarydifference from the configuration of FIG. 3 lies in the use ofadditional lenses (lens 430 and/or lens 440) which help to relay theapertures of a microlens array (MLA) 420 and creates a telecentric 3Dlightfield. (R. Martinez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, andM. Martínez-Corral, “Enhanced viewing-angle integral imaging bymultiple-axis telecentric relay system,” Optics Express, Vol. 15, Issue24, pp. 16255-16260, 21 Nov. 2007.) Lens 430 and lens 440 have the samefocal distance, f₁=f₂, with lens 430 directly attached to the MLA 420and lens 440 placed at a focal distance, f₁, away. The gap between themicrodisplay 410 and the MLA 420 is the same as the focal distance, f₀,of the MLA 420. The main advantages of this alternative design are thepotential increase of viewing angle for the reconstructed 3D scene,compactness, ease of integration with the HMD viewing optics, andblocking of the flipped images created by rays refracted by microlenses421 of the MLA 420 other than the correctly paired elemental image 401and microlens 421.

Although the InI method is promising, improvements are still desirabledue to three major limitations: (1) low lateral and longitudinalresolutions; (2) narrow depth of field (DOF); and (3) limited field ofview angle. These limitations are subject to the limited imagingcapability and finite aperture of microlenses, poor spatial resolutionof large-size displays, and the trade-off relationship between wide viewangle and high spatial resolution. Conventional InI systems typicallyyield low lateral and depth resolutions and narrow DOF. Theselimitations, however, can be alleviated in a wearable InI-HMD system ofthe present invention. First, microdisplays with large pixel counts andvery fine pixels (e.g. ˜5 μm pixel size) may be used in the presentinvention to replace large-pixel display devices (˜200-500 μm pixelsize) used in conventional InI displays, offering at least 50× gain inspatial resolution, FIG. 7. Secondly, due to the nature of HMD systems,the viewing zone is well confined and therefore a much smaller number ofelemental images would be adequate to generate the full-parallaxlightfields for the confined viewing zone than large-sizeauto-stereoscopic displays. Thirdly, to produce a perceived 3D volumespanning from 40 cm to 5 m depth range in an InI-HMD system, a verynarrow depth range (e.g. Z_(o)˜3.5 mm) is adequate for the intermediate3D scene reconstructed by the micro-InI unit, which is much moreaffordable than in a conventional stand-alone InI display systemrequiring at least 50 cm depth range to be usable, FIG. 7. Finally, byoptimizing the microlenses and the HMD viewing optics together, thedepth resolution of the overall InI-HMD system can be substantiallyimproved, overcoming the imaging limit of a stand-alone InI system.

The lightfields of the miniature 3D scene reconstructed by a micro-InIunit may be relayed by eyepiece optics into the eye for viewing. Theeyepiece optics not only effectively couples the 3D lightfields into theeye (exit) pupil but may also magnify the 3D scene to create a virtual3D display appearing to be at a finite distance from the viewer.

As an example, FIG. 5 schematically illustrates the integration of amicro-InI unit 530 with conventional eyepiece optics 540. The micro-InIunit 530 may include a microdisplay 510 and microlens array 520 that maybe configured in a similar manner to that illustrated in FIG. 3. Themicro-InI unit 530 reconstructs a miniature 3D scene (located at AOB inFIG. 5) which is located near the back focal point of the eyepieceoptics 540. Through the eyepiece optics 540 the miniature scene may bemagnified into an extended 3D display at A′O′B′ which can then be viewedfrom a small zone constrained by the exit pupil of the eyepiece optics540. Due to the 3D nature of the reconstructed scene, a differentviewing perspective is seen at different locations within the exitpupil.

Among the different methods for HMD designs, freeform optical technologydemonstrates great promise in designing compact HMD systems. FIG. 6Aillustrates the schematics of an exemplary configuration of a wearable3D augmented reality display 600 in accordance with the presentinvention. The wearable 3D augmented reality display 600 includes a 3DInI unit 630 and a freeform eyepiece 640. The micro-InI unit 630 mayinclude a microdisplay 610 and microlens array 620 that may beconfigured in a similar manner to that illustrated in FIG. 3. Thisconfiguration 600 adopts a wedge-shaped freeform prism as the eyepiece640, through which the 3D scene reconstructed by the micro-InI unit 630is magnified and viewed. Such eyepiece 640 is formed by three freeformoptical surfaces which are labeled as 1, 2, and 3, respectively, whichmay be rotationally asymmetric surfaces. The exit pupil is where the eyeis placed to view the magnified 3D scene, which is located of thevirtual reference plane conjugate to the reference plane of the 3D InIunit 630. A light ray emitted from a 3D point (e.g. A) located at theintermediate scene is first refracted by the surface 3 of the freeformeyepiece 640 located closest to the reference plane. Subsequently, thelight ray experiences two consecutive reflections by the surfaces 1′ and2, and finally is transmitted through the surface 1 and reaches the exitpupil of the system. Multiple ray directions from the same object point(e.g. each of the 3 rays from point A), each of which represents adifferent view of the object, impinge on different locations of the exitpupil and reconstruct a virtual 3D point (e.g. A′) in front of the eye.

Rather than requiring multiple elements, the optical path is naturallyfolded within a three-surface prism structure of the eyepiece 640, whichhelps reduce the overall volume and weight of the optics substantiallywhen compared with designs using rotationally symmetric elements.

To enable see-through capability for AR systems, surface 2 of theeyepiece 640 may be coated as a beam splitting mirror. A freeformcorrector lens 650 may be added to provide a wearable 3D augmentedreality display 690 having improved see-through capability. Thecorrector lens 650 may include two freeform surfaces which may beattached to the surface 2 of the eyepiece 640 to correct the viewingaxis deviation and undesirable aberrations introduced by the freeformprism eyepiece 640 to the real world scene. The rays from the virtuallightfield generated by the 3D InI unit 630 are reflected by surface 2of the prism eyepiece 640, while the rays from a real-world scene aretransmitted through the freeform eyepiece 640 and corrector lens 650,FIG. 6C. FIG. 6C schematically illustrates the integration andraytracing of the overall wearable 3D augmented reality display 690. Thefront surface of the freeform corrector lens 650 matches the shape ofsurface 2 of the prism eyepiece 640. The back surface 4 of the correctorlens 650 may be optimized to minimize the shift and distortionintroduced to the rays from a real-world scene when the corrector lens650 is combined with the prism eyepiece 640. The additional correctorlens 650 is not expected to noticeably increase the footprint and weightof the overall system 690.

Thus, in devices of the present invention, the freeform eyepiece 640 mayimage the lightfield of a 3D surface AOB, rather than a 2D imagesurface. In such an InI-HMD system 600, 690, the freeform eyepiece 640can reconstruct the lightfield of a virtual 3D object A′O′B′ at alocation optically conjugate to the lightfield of a real object, whilein a conventional HMD system the eyepiece creates a magnified 2D virtualdisplay which is optically conjugate to the 2D microdisplay surface.

In another of its aspects, the present invention may provide a 3Daugmented reality optical see-through HMD 700 in which the location ofthe virtual reference plane may be adjusted, FIG. 6D. The ability toadjust the location of the virtual reference plane may be particularlyuseful in addressing the accommodation-convergence discrepancy byselecting the location of the virtual reference plane containing theaugmented reality information relative to the location of objects in thereal world scene being observed by the viewer. In this regard, FIG. 6Dschematically illustrates an alternative configuration of a 3D augmentedreality optical see-through HMD 700 in accordance with the presentinvention which integrates a vari-focal element 728 in the optical pathof a micro-InI module 730. As with the design of FIG. 6C, the HMD 700may include an eyepiece 640, corrector lens 650, and a micro-InI unit730 with a microdisplay 710 and microlens array 720, which may beidentical to those used in FIG. 6C.

The optical power of the vari-focal element 728 (VFE) may be varied byapplying an electrical voltage to the element. Examples of vari-focalelements 728 that may be used include liquid crystal lenses, liquidlenses, membrane mirrors, or birefringent lenses. Having the ability todynamically change the optical power of the micro-InI unit 730 not onlyallows dynamic control of the axial position of the reference plane ofthe 3D reconstructed scene, but also enables dynamic adjustment of theview angle of the reconstructed scene. For instance, varying the voltageon the VFE 728 allows one to place the virtual reference plane at adistance as close as 25 cm (e.g. at O′) or as far as optical infinity(e.g. at O″) or vice versa without having to make mechanical changes.Varying the optical power of the VFE 728 also enables dynamic control onthe ray angles of the light reconstructing the 3D scene and thuscontrols the viewing angle. This capability enabled through the VFE 728allows one to improve the longitudinal resolution, extend the depth offield, and increase viewing angles. The change of optical power on theVFE 728 may be controlled by the knowledge of the 3D scene depth to berendered, or the knowledge of the user's region of interest. Forinstance, knowing the absolute depth of the reconstructed virtual scene(e.g. A′O′B′) with respect to the viewer and the depth range of thescene allows the system to properly position the virtual reference planewith optimal longitudinal resolution and viewing angle. Alternatively,the region of interest of the viewer, which may be detected through aneye movement tracking device, may be dynamically acquired and utilizedto position the virtual reference plane accordingly. When both the VFE728 and the microdisplay 710 operate at high speed (e.g., at least twicethe critical flickering frequency of the human visual system), 3D scenesof different depths can be rendered time sequentially with the referenceplane placed over these depths to cover an extended depth volume. Thetime-sequentially rendered scenes may then be viewed as a continuous 3Dvolume due to the advantage of the speed. A liquid crystal lens 728 maybe used for varying the depth of field of the integral imaging display.Other types of spatial light modulators may be used as well such asdeformable mirror devices for high speed modulation.

In yet another of its aspects, the present invention may provide a 3Daugmented reality optical see-through HMD 800 which integratesthree-dimensional (3D) or 2D object recognition capability, FIG. 6E. Forexample, FIG. 6E schematically illustrates an alternative configurationof a 3D augmented reality optical see-through HMD 800 in accordance withthe present invention which integrates three-dimensional (3D) or 2Dobject recognition capability with augmented reality (AR). As with thedesign of FIG. 6C, the HMD 800 may include an eyepiece 640, correctorlens 650, and a micro-InI unit 830 with a microdisplay 810 and microlensarray 820, which may be identical to those used in FIG. 6C. The HMD 800may combine an integral imaging optical 3D display method and 2D/3Dobject recognition capability by attaching at least one camera 815 tothe HMD 800, FIG. 6E. The current state of the art allows theavailability of light weight mega pixel cameras as small as onemillimeter square. (Sony 960H Micro Pinhole Covert Camera #20-100.) Thecamera 815 may be a regular video camera, but more importantly it can bea 3D integral imaging image capture system where a micro lenslet arrayis utilized in the imaging optics to capture the 3D lightfield of thereal-world scene 10. Using the images captured by such a camera 815, onecan implement either 2D imaging or 3D integral imaging acquisition,numerical visualization, and 2D or 3D object recognition, segmentation,and localization of objects in the scene 10. While the real scene 10 isobserved by the user directly through the free form optics 640, 650,object recognition capability enabled by the capture unit 850 can allowthe intelligent extraction of information about the detected objects andscene, their description, their position, their range using (3D imagingproperties of integral imaging), their relation with other objects inthe scene, etc and such information can be presented to the viewerthrough augmenting of the reality. For example, if the viewer is lookingfor a particular object in a crowded scene, the recognition capabilitycan detect this object and present it, including its location to theviewer using AR. A variety of architectures can be used for the 3Dobject visualization, which may be run on a general purpose computer toprovide the capture unit 850. (S. Hong, J. Jong, and B. Javidi,“Three-dimensional volumetric object reconstruction using computationalintegral imaging,” Journal of Optics Express, on-line Journal of theOptical Society of America, Vol. 12, No. 3, pp. 483-491, Feb. 9, 2004.R. Schulein, M. DaneshPanah, and B. Javidi, “3D imaging with axiallydistributed sensing,” Journal of Optics Letters, Vol. 34, Issue 13, pp.2012-2014, 1 Jul. 2009) or recognition. (S. Kishk and B. Javidi,“Improved Resolution 3D Object Sensing and Recognition using timemultiplexed Computational Integral Imaging,” Optics Express, on-lineJournal of the Optical Society of America, vol. 11, no. 26, pp.3528-3541, Dec. 29, 2003. R. Schulein, C. Do, and B. Javidi,“Distortion-tolerant 3D recognition of underwater objects using neuralnetworks,” Journal of Optical Society of America A, vol. 27, no. 3, pp461-468, March 2010. C. Manh Do, R. Martínez-Cuenca, and B. Javidi,“Three-dimensional object-distortion-tolerant recognition for integralimaging using independent component analysis,” Journal of OpticalSociety of America A 26, issue 2, pp 245-251 (1 Feb. 2009). S. Hong andB. Javidi, “Distortion-tolerant 3D recognition of occluded objects usingcomputational integral imaging,” Journal of Optics Express, Vol. 14,Issue 25, pp. 12085-12095, Dec. 11, 2006.). A variety of algorithms canbe used for 2D or 3D object recognition. (F. Sadjadi and B. Javidi,“Physics of Automatic Target Recognition,” Springer-Verlag, New York,2007. R. J. Schalkoff, Pattern Recognition: Statistical, Structural andNeural Approaches (Wiley, 1991). Christopher M. Bishop, Neural Networksfor Pattern Recognition, Oxford University Press, Inc. New York, NY1995. A. K. Jain, Fundamentals of Digital Image Processing, PrenticeHall. M. Daneshpanah, B. Javidi, and E. Watson, “Three dimensionalintegral imaging with randomly distributed sensors,” Journal of OpticsExpress, Vol. 16, Issue 9, pp. 6368-6377, Apr. 21, 2008. R. Schulein, M.DaneshPanah, and B. Javidi, “3D imaging with axially distributedsensing,” Journal of Optics Letters, Vol. 34, Issue 13, pp. 2012-2014, 1Jul. 2009). It is possible to have a 3D object recognition capability byusing a single camera 815 when the viewer is moving. Or, it is possibleto have a 3D object recognition capability by using a single camera 815with zero parallax by using axially distributed sensing when the vieweris moving towards the object. Examples

A proof-of-concept monocular prototype of an InI OST-HMD according tothe configuration of FIG. 6C was implemented using off-the-shelf opticalcomponents, FIG. 8. A micro-lens array (MLA) of a 3.3 mm focal lengthand 0.985 mm pitch was utilized. (These types of microlenses can bepurchased from Digital Optics Corp, SUSS Microoptics, etc.) Themicrodisplay was a 0.8″ organic light emitting display (OLED), whichoffered 1920×1200 color pixels with a pixel size of 9.6 nm. (EMA-100820,by eMagin Corp, Bellevue, Wash.) A freeform eyepiece along with asee-through corrector were used of the type disclosed in InternationalPatent Application No. PCT/US2013/065422, the entire contents of whichare incorporated herein by reference. The specifications of the eyepiece640 and corrector 650 are provided in the tables below. The eyepieceoffered a field of view of 40 degrees and approximately a 6.5 mm eyebox.Due to the strict telecentricity of the eyepiece design, it was adaptedto the InI setup with reasonably low crosstalk but with a narrow viewingzone. It is worth noting that adapting this particular freeform eyepiecedesign is not required for implementing the optical method described inthis invention. Alternative eyepieces may be designed and optimized forthis purpose.

System Prescription for Display Path

In Error! Reference source not found.1, surfaces #2-#4 specify thefree-form eyepiece 640. Table 1 surfaces #2 and #4 represent the samephysical surface and corresponds to eyepiece surface 1, in FIGS. 6A-6C.Table 1 surface #3 is corresponds eyepiece surface 2, and Table 1surface #5 corresponds to eyepiece surface 3, in FIGS. 6A-6C.

TABLE 1 Surface prescription of eyepiece - AR display path. SurfaceSurface Refract No. Type Y Radius Thickness Material Mode 1 (Stop)Sphere Infinity 0.000 Refract 2 XY Poly −185.496 0.000 PMMA Refract 3 XYPoly  −67.446 0.000 PMMA Reflect 4 XY Poly −185.496 0.000 PMMA Reflect 5XY Poly −830.046 0.000 Refract 6 Sphere Infinity 0.000 Refract

TABLE 2 System prescription for see-through path. Surface Surface Thick-Refract No. Type Y Radius X Radius ness Material Mode 1 (Stop) SphereInfinity Infinity 0.000 Refract 2 XY Poly −185.496  −185.496  0.000 PMMARefract 3 XY Poly −67.446 −67.446 0.000 PMMA Refract 4 XY Poly −67.446−67.446 0.000 PMMA Refract 5 XY Poly −87.790 −87.790 10.00 Refract 6Cylin- Infinity −103.400  6.5 NBK7 Refract drical 7 Sphere InfinityInfinity 0.000 Refract

System Prescription for Optical See-Through Path

In Table 2 surfaces #2 and #3 are eyepiece surfaces 1 and 3, modeled thesame as in the display path. Surfaces #4, #5 specify the freeformcorrector lens 650. Surface #4 is an exact replica of Surface #3(eyepiece surface 2).

TABLE 3 Optical surface prescription of Surface #2 and #4 of Table 1. YRadius −1.854965E+02  Conic Constant −2.497467E+01  X 0.000000E+00 Y0.000000E+00 X**2 −2.331157E−03  X * Y 0.000000E+00 Y**2 6.691726E−04X**3 0.000000E+00 X**2 * Y −1.066279E−04  X Y**2 0.000000E+00 Y**3−2.956368E−05  X**4 −1.554280E−06  X**3 * Y 0.000000E+00 X**2 * Y**21.107189E−06 X * Y**3 0.000000E+00 Y**4 1.579876E−07 X**5 0.000000E+00X**4 * Y 1.789364E−07 X**3 * Y**2 0.000000E+00 X**2 * Y**3−2.609879E−07  X * Y**4 0.000000E+00 Y**5 −6.129549E−10  X**6−3.316779E−08  X**5 * Y 0.000000E+00 X**4 * Y**2 9.498635E−09 X**3 *Y**3 0.000000E+00 X**2 * Y**4 9.042084E−09 X * Y**5 0.000000E+00 Y**6−4.013470E−10  X**7 0.000000E+00 X**6 * Y −8.112755E−10  X**5 * Y**20.000000E+00 X**4 * Y**3 1.251040E−09 X**3 * Y**4 0.000000E+00 X**2 *Y**5 −1.505674E−10  X * Y**6 0.000000E+00 Y**7 −4.419392E−11  X**84.236650E−10 X**7 * Y 0.000000E+00 X**6 * Y**2 −1.079269E−10  X**5 *Y**3 0.000000E+00 X**4 * Y**4 −1.678245E−10  X**3 * Y**5 0.000000E+00X**2 * Y**6 2.198604E−12 X * Y**7 0.000000E+00 Y**8 −2.415118E−12  X**90.000000E+00 X**8 * Y 4.113054E−12 X**7 * Y**2 0.000000E+00 X**6 * Y**3−1.805964E−12  X**5 * Y**4 0.000000E+00 X**4 * Y**5 9.480632E−13 X**3 *Y**6 0.000000E+00 X**2 * Y**7 2.891726E−13 X * Y**8 0.000000E+00 Y**9−2.962804E−14  X**10 −6.030361E−13  X**9 * Y 0.000000E+00 X**8 * Y**2−7.368710E−13  X**7 * Y**3 0.000000E+00 X**6 * Y**4 9.567750E−13 X**5 *Y**5 0.000000E+00 X**4 * Y**6 4.280494E−14 X**3 * Y**7 0.000000E+00X**2 * Y**8 −7.143578E−15  X * Y**9 0.000000E+00 Y**10 3.858414E−15N-Radius 1.000000E+00

TABLE 4 Decenter of Surface #2 and #4 of Table 1, relative to Surface #1of Table 1. Y DECENTER Z DECENTER ALPHA TILT 6.775E+00 2.773E+017.711E+00

TABLE 5 Optical surface prescription of Surface #3 of Table 1. Y Radius−6.744597E+01  Conic Constant −1.258507E+00  X 0.000000E+00 Y0.000000E+00 X**2 −1.300207E−03  X * Y 0.000000E+00 Y**2 4.658585E−04X**3 0.000000E+00 X**2 * Y −1.758475E−05  X Y**2 0.000000E+00 Y**3−1.684923E−06  X**4 −1.463720E−06  X**3 * Y 0.000000E+00 X**2 * Y**2−1.108359E−06  X * Y**3 0.000000E+00 Y**4 −1.098749E−07  X**50.000000E+00 X**4 * Y −7.146353E−08  X**3 * Y**2 0.000000E+00 X**2 *Y**3 −1.150619E−08  X * Y**4 0.000000E+00 Y**5 5.911371E−09 X**6−5.406591E−10  X**5 * Y 0.000000E+00 X**4 * Y**2 −1.767107E−09  X**3 *Y**3 0.000000E+00 X**2 * Y**4 −7.415334E−10  X * Y**5 0.000000E+00 Y**6−5.442400E−10  X**7 0.000000E+00 X**6 * Y 6.463414E−10 X**5 * Y**20.000000E+00 X**4 * Y**3 1.421597E−10 X**3 * Y**4 0.000000E+00 X**2 *Y**5 −3.464751E−11  X * Y**6 0.000000E+00 Y**7 −8.246179E−12  X**8−2.087865E−11  X**7 * Y 0.000000E+00 X**6 * Y**2 2.845323E−11 X**5 *Y**3 0.000000E+00 X**4 * Y**4 −5.043398E−12  X**3 * Y**5 0.000000E+00X**2 * Y**6 2.142939E−14 X * Y**7 0.000000E+00 Y**8 1.607499E−12 X**90.000000E+00 X**8 * Y −1.922597E−12  X**7 * Y**2 0.000000E+00 X**6 *Y**3 1.100072E−13 X**5 * Y**4 0.000000E+00 X**4 * Y**5 −4.806130E−14 X**3 * Y**6 0.000000E+00 X**2 * Y**7 −2.913177E−14  X * Y**80.000000E+00 Y**9 9.703717E−14 X**10 2.032150E−13 X**9 * Y 0.000000E+00X**8 * Y**2 −1.037107E−13  X**7 * Y**3 0.000000E+00 X**6 * Y**43.602862E−14 X**5 * Y**5 0.000000E+00 X**4 * Y**6 −8.831469E−15  X**3 *Y**7 0.000000E+00 X**2 * Y**8 2.178095E−15 X * Y**9 0.000000E+00 Y**101.784074E−15 N-Radius 1.000000E+00

TABLE 6 Decenter of Surface #3 of Table 5 relative to Surface #1 ofTable 1. Y DECENTER Z DECENTER ALPHA TILT 1.329E+01 4.321E+01 −8.856E+00

TABLE 7 Optical surface prescription of Surface #5 of Table 1. Y Radius−8.300457E+02  Conic Constant −9.675799E+00  X 0.000000E+00 Y0.000000E+00 X**2 −1.798206E−04  X * Y 0.000000E+00 Y**2 −2.606383E−03 X**3 0.000000E+00 X**2 * Y −7.767146E−05  X Y**2 0.000000E+00 Y**3−8.958581E−05  X**4 1.978414E−05 X**3 * Y 0.000000E+00 X**2 * Y**22.081156E−05 X * Y**3 0.000000E+00 Y**4 −1.073001E−06  X**5 0.000000E+00X**4 * Y 2.585164E−07 X**3 * Y**2 0.000000E+00 X**2 * Y**3−2.752516E−06  X * Y**4 0.000000E+00 Y**5 −1.470053E−06  X**6−1.116386E−07  X**5 * Y 0.000000E+00 X**4 * Y**2 −3.501439E−07  X**3 *Y**3 0.000000E+00 X**2 * Y**4 1.324057E−07 X * Y**5 0.000000E+00 Y**6−9.038017E−08  X**7 0.000000E+00 X**6 * Y 3.397174E−10 X**5 * Y**20.000000E+00 X**4 * Y**3 −1.873966E−08  X**3 * Y**4 0.000000E+00 X**2 *Y**5 4.051880E−08 X * Y**6 0.000000E+00 Y**7 −3.973293E−09  X**8−1.881791E−10  X**7 * Y 0.000000E+00 X**6 * Y**2 5.519986E−09 X**5 *Y**3 0.000000E+00 X**4 * Y**4 3.822268E−09 X**3 * Y**5 0.000000E+00X**2 * Y**6 −3.024448E−09  X * Y**7 0.000000E+00 Y**8 2.673713E−11 X**90.000000E+00 X**8 * Y 1.006915E−10 X**7 * Y**2 0.000000E+00 X**6 * Y**3−2.945084E−10  X**5 * Y**4 0.000000E+00 X**4 * Y**5 5.958040E−10 X**3 *Y**6 0.000000E+00 X**2 * Y**7 −3.211903E−10  X * Y**8 0.000000E+00 Y**92.296303E−11 X**10 5.221834E−12 X**9 * Y 0.000000E+00 X**8 * Y**21.135044E−11 X**7 * Y**3 0.000000E+00 X**6 * Y**4 −1.050621E−10  X**5 *Y**5 0.000000E+00 X**4 * Y**6 5.624902E−11 X**3 * Y**7 0.000000E+00X**2 * Y**8 5.369592E−12 X * Y**9 0.000000E+00 Y**10 2.497657E−12N-Radius 1.000000E+00

TABLE 8 Decenter of Surface #5 relative to Surface #1 of Table 1. YDECENTER Z DECENTER ALPHA TILT .427E+01 3.347E+01 7.230E+01

TABLE 9 Optical surface prescription of Surface #5 of Table 2. Y Radius−8.779024E+01  Conic Constant −7.055198E+00  X 0.000000E+00 Y0.000000E+00 X**2 −3.191225E−03  X * Y 0.000000E+00 Y**2 4.331992E−03X**3 0.000000E+00 X**2 * Y −9.609025E−05  X Y**2 0.000000E+00 Y**3−2.432809E−05  X**4 −2.955089E−06  X**3 * Y 0.000000E+00 X**2 * Y**22.096887E−07 X * Y**3 0.000000E+00 Y**4 −9.184356E−07  X**5 0.000000E+00X**4 * Y 3.707556E−08 X**3 * Y**2 0.000000E+00 X**2 * Y**3−1.535357E−07  X * Y**4 0.000000E+00 Y**5 −1.445904E−08  X**6−4.440851E−09  X**5 * Y 0.000000E+00 X**4 * Y**2 1.686424E−09 X**3 *Y**3 0.000000E+00 X**2 * Y**4 6.770909E−09 X * Y**5 0.000000E+00 Y**6−3.713094E−10  X**7 0.000000E+00 X**6 * Y −1.316067E−10  X**5 * Y**20.000000E+00 X**4 * Y**3 7.924387E−10 X**3 * Y**4 0.000000E+00 X**2 *Y**5 −8.011955E−11  X * Y**6 0.000000E+00 Y**7 3.606142E−11 X**83.208020E−11 X**7 * Y 0.000000E+00 X**6 * Y**2 −2.180416E−11  X**5 *Y**3 0.000000E+00 X**4 * Y**4 −3.616135E−11  X**3 * Y**5 0.000000E+00X**2 * Y**6 −5.893434E−12  X * Y**7 0.000000E+00 Y**8 3.081069E−12 X**90.000000E+00 X**8 * Y 1.267096E−12 X**7 * Y**2 0.000000E+00 X**6 * Y**3−1.848104E−12  X**5 * Y**4 0.000000E+00 X**4 * Y**5 5.208420E−14 X**3 *Y**6 0.000000E+00 X**2 * Y**7 1.198597E−13 X * Y**8 0.000000E+00 Y**9−6.834914E−14  X**10 −1.706677E−14  X**9 * Y 0.000000E+00 X**8 * Y**2−1.614840E−14  X**7 * Y**3 0.000000E+00 X**6 * Y**4 8.739087E−14 X**5 *Y**5 0.000000E+00 X**4 * Y**6 3.940903E−15 X**3 * Y**7 0.000000E+00X**2 * Y**8 5.435162E−15 X * Y**9 0.000000E+00 Y**10 −2.259169E−15 N-Radius 1.000000E+00

TABLE 10 Decenter of Surface #5 relative to Surface #1 of Table 2. YDECENTER Z DECENTER ALPHA TILT 3.358E+00 4.900E+01 6.765E+00

As used in the system prescription Tables, e.g., Table 1 or Error!Reference source not found.2, the term “XY Poly” refers to a surfacewhich may be respresented by the equation

${z = {{\frac{{cr}^{2}}{1 + \sqrt{1 - {\left( {1 + k} \right)c^{2}r^{2}}}} + {\sum\limits_{j = 2}^{66}{C_{j}x^{m}y^{n}\mspace{31mu} j}}} = {\frac{\left( {m + n} \right)^{2} + m + {3n}}{2} + 1}}},$

where z is the sag of the free-form surface measured along the z-axis ofa local x, y, z coordinate system, c is the vertex curvature (CUY), r isthe radial distance, k is the conic constant, and C_(j) is thecoefficient for x^(m)y^(n).

For demonstration purposes, a 3D scene including a number “3” and aletter “D” was simulated. In the visual space, the objects “3” and “D”were located ˜4 meters and 30 cms away from the eye position,respectively. To clearly demonstrate the effects of focusing, thesecharacter objects, instead of using plain solid colors, were renderedwith black line textures. An array of 18×11 elemental images of the 3Dscene were simulated (FIG. 9), each of which consisted of 102 by 102color pixels. The 3D scene reconstructed by the micro-InI unit wasapproximately 10 mm away from the MLA and the separation of the tworeconstructed targets was approximately 3.5 mm in depth in theintermediate reconstruction space.

FIGS. 10A through 10D shows a set of images captured with a digitalcamera placed t the eye position. To demonstrate the effects of focusand see-through view, in the real-world view, a Snellen letter chart anda printed black-white grating target were placed ˜4 meters and 30 cmaway from the viewer, respectively, which corresponded to the locationsof the objects “3” and “D”, respectively.

FIGS. 10A and 10B demonstrate the effects of focusing the camera on theSnellen chart and grating target, respectively. The object “3” appearedto be in sharp focus when the camera was focused on the far Snellenchart while the object “D” was in focus when the camera was focused onthe near grating target. FIGS. 10C and 10D demonstrate the effects ofshifting the camera position from the left to the right sides of theeyebox while the camera focus was set on the near grating target. Asexpected, slight perspective change was observed between these twoviews. Although artifacts admittedly are visible and further developmentis needed, the results clearly demonstrated that the proposed method forAR display can produce correct focus cues and true 3D viewing in a largedepth range.

The invention described and claimed herein is not to be limited in scopeby the specific embodiments herein disclosed, since these embodimentsare intended as illustrations of several aspects of the invention. Anyequivalent embodiments are intended to be within the scope of thisinvention. Indeed, various modifications of the invention in addition tothose shown and described herein will become apparent to those skilledin the art from the foregoing description. Such modifications are alsointended to fall within the scope of the appended claims.

A number of patent and non-patent publications are cited in thespecification, the entire disclosure of each of these publications isincorporated by reference herein.

REFERENCES

-   [1] Yano, S., Emoto, M., Mitsuhashi, T., and Thwaites, H., “A study    of visual fatigue and visual comfort for 3D HDTV/HDTV images,”    Displays, 23(4), pp. 191-201, 2002.-   [2] S. J. Watt, K. Akeley, M. O. Ernst, and M. S. Banks, “Focus Cues    Affect Perceived Depth,” J. Vision, 5(10), 834-862, (2005).-   [3] D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks,    “Vergence-Accommodation Conflicts Hinder Visual Performance and    Cause Visual Fatigue,” J. Vision, 8(3), 1-30, (2008).-   [4] G. Lippmann, “Epreuves reversibles donnant la sensation du    relief,” Journal of Physics (Paris) 7, 821-825 (1908).-   [5] C. B. Burckhardt, “Optimum parameters and resolution limitation    of integral photography,” J. Opt. Soc. Am. 58, 71-76 (1968).-   [6] T. Okoshi, “Optimum design and depth resolution of lens-sheet    and projection-type three-dimensional displays,” Appl. Opt. 10,    2284-2291 (1971).-   [7] F. Okano, H. Hoshino, J. Arai y I. Yuyama, “Real-time pickup    method for a three-dimensional image based on integral photography,”    Appl. Opt. 36, 1598-1603 (1997).-   [8] J. Aran, “Depth-control method for integral imaging,” Optics    Letters, 33(3): 279-282, 2008.-   [9] H. Hua, “Sunglass-like displays become a reality with freeform    optical technology,” SPIE Newsroom, 2012.-   [10] H. Hua and C. Gao, A compact, eye-tracked optical see-through    head-mounted display, Proc. SPIE 8288, p. 82881F, 2012.-   [11] H. Hua, X. Hu, and C. Gao, “A high-resolution optical    see-through head-mounted display with eyetracking capability,”    Optics Express, November 2013.-   [12] D. Cheng, Y.Wang, H. Hua, and M. M. Talha, Design of an optical    see-through headmounted display with a low f-number and large field    of view using a free-form prism, App. Opt. 48 (14), pp. 2655-2668,    2009.-   [13] D. Cheng, Y.Wang, H. Hua, and J. Sasian, Design of a    wide-angle, lightweight headmounted display using free-form optics    tiling, Opt. Lett. 36 (11), pp. 2098-2100, 2011.-   [14] A. Okuyama and S. Yamazaki, Optical system and image observing    apparatus and image pickup apparatus using it, U.S. Pat. No.    5,706,136, 1998.-   [15] S. Yamazaki, K. Inoguchi, Y. Saito, H. Morishima, and N.    Taniguchi, Thin widefield-of-view HMD with free-form-surface prism    and applications, Proc. SPIE 3639, p. 453, 1999.-   [16] A. Jones, I. McDowall, Yamada H., M. Bolas, P. Debevec,    Rendering for an Interactive 360° Light Field Display ACM    Transactions on Graphics (TOG)—Proceedings of ACM SIGGRAPH 2007,    26(3), 2007.-   [17] Tibor Balogh, “The HoloVizio System,” Proceedings of SPIE, VO1    6055, 2006.-   [18] Y. Takaki, Y. Urano, S. Kashiwada, H. Ando, and K. Nakamura,    “Super multi-view winshield display for long-distance image    information presentation,” Opt. Express, 19, 704-16, 2011.-   [19] Blundell, B. G., and Schwarz, A. J., “The classification of    volumetric display systems: characteristics and predictability of    the image space,” IEEE Transaction on Visualization and Computer    Graphics, 8(1), pp. 66-75, 2002.-   [20] P. A. Blanche, et al, “Holographic three-dimensional    telepresence using large-area photorefractive polymer”, Nature, 468,    80-83, November 2010.-   [21] Rolland, J. P., Kureger, M., and Goon, A., “Multifocal planes    head-mounted displays,” Applied Optics, 39(19), pp. 3209-14, 2000.-   [22] Akeley, K., Watt, S., Girshick, A., and Banks, M., “A stereo    display prototype with multiple focal distances,” Proc. of SIGGRAPH,    pp. 804-813, 2004.-   [23] Schowengerdt, B. T., and Seibel, E. J., “True 3-D scanned voxel    displays using single or multiple light sources,” Journal of SID,    14(2), pp. 135-143, 2006.-   [24] S. Liu, H. Hua, D. Cheng, “A Novel Prototype for an Optical    See-Through Head-Mounted Display with Addressable Focus Cues,” IEEE    Transactions on Visualization and Computer Graphics, 16(3), 381-393,    (2010).-   [25] S. Liu and H. Hua, “A systematic method for designing    depth-fused multi-focal plane three-dimensional displays,” Opt.    Express, 18, 11562-11573, (2010)-   [26] X. Hu and H. Hua, “Design and assessment of a depth-fused    multi-focal-plane display prototype,” Journal of Display Technology,    December 2013.-   [27] Suyama, S., Ohtsuka, S., Takada, H., Uehira, K., and Sakai, S.,    “Apparent 3D image perceived from luminance-modulated two 2D images    displayed at different depths,” Vision Research, 44: 785-793, 2004.-   [28] J. Hong, S. Min, and B. Lee, “Integral floating display systems    for augmented reality,” Applixed Optics, 51(18):4201-9, 2012.-   [29] A. Malmone, and H. Fuchs, “Computational augmented reality    eyeglasses,” Proc. of ISMAR 2012.-   [30] Rolland, J. P., and Hua, H., “Head-mounted display systems,” in    Encyclopedia of Optical Engineering (Editors: R. Barry Johnson and    Ronald G. Driggers), New York, NY: Marcel Dekker, pp. 1-13, 2005.-   [31] H. Mukawa, K. Akutsu, I. Matsumura, S. Nakano, T. Yoshida, M.    Kuwahara, and K. Aiki, A full-color eyewear display using planar    waveguides with reflection volume holograms, J. Soc. Inf. Display 19    (3), pp. 185-193, 2009.-   [32] http://www.lumus-optical.com/-   [33] http://www.innovega-inc.com-   [34] http://www.epson.com/cgi-bin/Store/jsp/Moverio/Home.do-   [35] http://www.google.com/glass/start/-   [36] M. Martínez-Corral, H. Navarro, R. Martínez-Cuenca, G.    Saavedra, and B. Javidi, “Full parallax 3-D TV with programmable    display parameters,” Opt. Phot. News 22, 50-50 (2011).-   [37] J. S. Jang and B. Javidi, “Large depth-of-focus    time-multiplexed three-dimensional integral imaging by use of    lenslets with non-uniform focal lengths and aperture sizes,” Opt.    Lett. vol. 28, pp. 1924-1926 (2003).-   [38] Chih-Wei Chen, Myungjin Cho, Yi-Pai Huang, and Bahram Javidi,    “Improved viewing zones for projection type integral imaging 3D    display using adaptive liquid crystal prism array,” IEEE Journal of    Display Technology, 2014.-   [39] Xiao Xiao, Bahram Javidi, Manuel Martinez-Corral, and Adrian    Stern, “Advances in Three-Dimensional Integral Imaging: Sensing,    Display, and Applications,” Applied Optics, 52(4): 546-560, 2013.-   [40] J. S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral    imaging with large depth of focus by use of real and virtual image    fields,” Opt. Lett. 28:1421-23, 2003.-   [41] S. Bagheri and B. Javidi, “Extension of Depth of Field Using    Amplitude and Phase Modulation of the Pupil Function,” Journal of    Optics Letters, vol. 33, no. 7, pp. 757-759, 1 Apr. 2008.

What is claimed is:
 1. A 3D augmented reality display, comprising: amicrodisplay for providing a virtual 3D image for display to a user;variable focus display optics configured to receive optical radiationfrom the microdisplay and configured to create a 3D lightfield from thereceived radiation at a location that may be varied by the variablefocus optics; and an eyepiece in optical communication with the displayoptics configured to receive the 3D lightfield from the display opticsand deliver the received radiation to an exit pupil of the augmentedreality display.
 2. The 3D augmented reality display of claim 1, whereinthe variable focus display optics includes one or more of a liquidcrystal lens, a liquid lens, a membrane mirror, or birefringent lens. 3.The 3D augmented reality display of claim 1, comprising an imagingdevice configured to receive information from a scene.
 4. A 3D augmentedreality display, comprising: a microdisplay for providing a virtual 3Dimage for display to a user; display optics configured to receiveoptical radiation from the microdisplay and configured to create a 3Dlightfield from the received radiation; an eyepiece in opticalcommunication with the display optics configured to receive the 3Dlightfield from the display optics and deliver the received radiation toan exit pupil of the augmented reality display; and an imaging deviceconfigured to receive information from a scene.
 5. The 3D augmentedreality display of claim 3, wherein the imaging device comprises acamera.
 6. The 3D imaging augmented reality display claim 3, wherein theimaging device comprises an integral imaging image capture system. 7.The 3D imaging augmented reality display of claim 3, wherein the imagingdevice comprises a 3D imaging capture system using axially distributedimaging.
 8. The 3D augmented reality display of claim 3, wherein theimaging device includes an information capture unit for analyzing theinformation received by the imaging device to perform object recognitionthereon.
 9. The 3D augmented reality display of claim 3, wherein theimaging device is disposed in communication with the microdisplay toprovide the analysis to the microdisplay.
 10. The 3D imaging augmentedreality display and object recognition of claim 3, wherein the imagecapture device includes a variable focal length optics to provideenhanced performance in terms of depth of field.
 11. The 3D augmentedreality display of claim 3, wherein the display optics comprisesintegral imaging optics.
 12. The 3D augmented reality display of claim3, wherein the eyepiece comprises a selected surface configured toreceive the 3D lightfield from the display optics and reflect thereceived radiation to the exit pupil, the selected surface alsoconfigured to receive optical radiation from a source other than themicrodisplay and to transmit the optical radiation to the exit pupil.13. The 3D augmented reality display of claim 3, wherein the eyepiececomprises a freeform prism shape.
 14. The 3D augmented reality displayof claim 3, wherein the eyepiece comprises a first surface configured toreceive and refract optical radiation from the display optics andcomprises a second surface configured to receive the refracted opticalradiation from the first surface, the second surface configured toreflect the optical radiation to a third surface of the eyepiece, thethird surface configured to reflect the optical radiation reflected fromthe second surface to the exit pupil.
 15. The 3D augmented realitydisplay of claim 4, comprising a corrector lens disposed adjacent thesecond surface of the eyepiece.
 16. The 3D augmented reality display ofclaim 3, wherein one or more of the surfaces of the eyepiece comprise arotationally asymmetric surface.
 17. The 3D augmented reality display ofclaim 3, wherein the eyepiece comprises a wedge shape.
 18. The 3Daugmented reality display of claim 3, wherein the eyepiece comprises asurface respresented by the equation${z = {{\frac{{cr}^{2}}{1 + \sqrt{1 - {\left( {1 + k} \right)c^{2}r^{2}}}} + {\sum\limits_{j = 2}^{66}{C_{j}x^{m}y^{n}\mspace{31mu} j}}} = {\frac{\left( {m + n} \right)^{2} + m + {3n}}{2} + 1}}},$where z is the sag of the free-form surface measured along the z-axis ofa local x, y, z coordinate system, c is the vertex curvature (CUY), r isthe radial distance, k is the conic constant, and C_(j) is thecoefficient for x^(m)y^(n).
 19. The 3D augmented reality display ofclaim 3, wherein the display optics comprises one or more of aholographic display, multi-layer computational lightfield display, and avolumetric display.
 20. The 3D imaging augmented reality display ofclaim 3, wherein the 3D lightfield provides full parallax.
 21. The 3Dimaging augmented reality display of claim 3, wherein the 3D lightfieldprovides partial parallax.